[POC] make it possible to _not_ have a baseline for git registries #1803
Conversation
strega-nil-re
changed the title
strega-nil-re
force-pushed
the
dev/strega-nil/push-sxpzwzowtzku
branch
from
c39ad47 to
acdf88a
Compare
|
@vicroms @JavierMatosD @AugP and I discussed this today. However, the folks present had not had time to completely internalize what the impact of this change would be. We're going to discuss next week as well. The observations below are mine, not speaking as the team.
Why can't you do this now? As long as the current baseline is older than any of the PRs the suggested version>= constraints will work correctly.
That's not correct though. It's true that baselines are linearized but it is not true that the versions in baselines are always increasing.
This problem is not actually fixed by this proposal as long as there is anything in the transitive closure of dependencies with a
It's possible that we would be happy to change this behavior without dropping the baselines requirement entirely. |
If the latter thing was to be an option, we could certainly accept that also.
This is definitely more of a training and pit of success thing. I think the versioning situation I am requesting is a better situation for internal libraries, and I hope more people use it. I think that giving documentation and help to this situation (and with stuff like depend-a-bot, external support) is beneficial. I am certainly willing to do a decent amount of work here, of course.
That is unfortunate.
While true, this is an incredibly confusing thing for people in my world. Baselines are consistently the most confusing thing we deal with with regard to vcpkg. |
Internally to reMarkable, we want to not have a baseline for our internal dependencies, and only use `"version>="` constraints; this current set of changes does work for our current workflow, but of course we can make the code better; I just want to put it out there to see if there's interest (and so we can maybe avoid a fork).
strega-nil-re
force-pushed
the
dev/strega-nil/push-sxpzwzowtzku
branch
from
acdf88a to
05b70f5
Compare
|
I'm taking some notes from a discussion between @AugP @vicroms @ras0219-msft (partially) @JavierMatosD @MahmoudGSaleh and I. The notes as written are my words though, please take them for the paraphrasing that they are. I note that while we have said we do think linearizing versions as the baseline mechanism does is how we believe versions should be used by most customers, we don't actually have much feedback from folks pushing the envelope with versioning so I think we should be willing to consider this even though we have rejected similar requests in the past now that this has been out for a while. If you try something like this internally with Remarkable as a case study that would be helpful in making this be acceptable to us. @vicroms observes:
@ras0219-msft notes that we don't fully understand how this proposal interacts with the:
As far as we understand it in step 5,
it isn't clear how the new version of scene you want to test with would be found as it is not yet present in the registry. It seems you would need an overlay port for this, and if that's the case there's no reason to make a baseline change.
Perhaps the problem is more a naming issue and "baseline" should be "version>=". @BillyONeal observes: At a minimum we're going to want explicit tests of this feature's behavior, and proposed docs changes to github.com/MicrosoftDocs/vcpkg-docs before we would push the merge button. However given that I think you were asking if we would reject this out of hand before doing that additional work, I hope the replies above make some sense. My main concerns remain the parts of the scenarios you want to work that aren't actually made to work by this change. @vicroms will add a reply under this one with additional thoughts |
|
I would love to hear from @vicroms, but I did want to say one thing:
We're hoping to also make all of our internal libraries use |
3 participants
RFC: Registries without a Baseline
Inside reMarkable, we've been using vcpkg for around five years now for
external, open source libraries, as well as for some internal libraries.
However, we are now working on using vcpkg for more internal libraries, with
significantly more inter-dependencies. This has resulted in our discovering
serious issues with how the baseline works in a world where vcpkg acts more like
a Cargo than an apt.
The Current Experience
Our current private registry set up is very similar to the public registry; we
keep the baseline up to date, we test all of the libraries at head against each
other, etc. This generally works okay, but it starts to fall apart when we have
dependencies between libraries. It is also deeply confusing to people; overrides
have especially caused significant confusion.
Example 1 – Breaking Changes are Difficult
We use vcpkg to download, build and install our dependencies for all of our
libraries. This means that when there is a breaking change in a
deep-in-the-stack library, it is very frustrating to make that change. As our
example, the deep-in-the-stack library would be scene, and the dependent library
would be gfx; our application is called xochitl.
CI failures from gfx
vcpkg-configuration.json, so that we can test in gfx's CI
can test in xochitl's CI
CI passes
A lot of this complexity is inherent, but a lot of it is because of baselines.
In my ideal world, we would be using versions, and the result would end up
looking more like:
test it in CI (note: no baseline change!)
CI
You will notice that without having to update all the baselines all the time,
there's no need to ignore CI so many times, and there's no need to keep updating
PRs with the "correct" baseline.
Example 2 - Overrides/Baselines are confusing
Now, this example might be more of an issue of training, but I think it really
shows how the baseline is confusing. The assumption by many people inside
reMarkable, before I came along and made it more clear how it actually worked,
was that the baseline must be moved forward when updating versions. This means
that whenever there was a new version of one library, the baseline moved the
versions forward for everyone.
We had an issue recently where a team released version 1.3.0 of scene (an
internal library); this was intended as a prerelease version, and was broken
with the version of gfx that was in the registry, but because we had the idea
that "the baseline must always be the latest version", the baseline was also
moved forward. The team involved attempted to "fix" the issue by setting an
override in gfx, "pushing back" the version used to 1.2.4; this, of course, did
not work since overrides are not transitive.
My desired outcome would instead be to use the version>= feature that we have.
My hope is that we would update versions when we need to. This would fit much
more closely with the mental model that people actually have.
This would work inside the current vcpkg, but the insistence on baselines does
result in serious confusion for people; I personally believe that allowing one
to completely remove the baseline would allow us to avoid the confusion
completely.
Example 3 - Baselines are Confusing (part 2. Electric Boogaloo)
Added on 14 October since it happened today.
It is very confusing to people that baselines are not equivalent to version sets.
Just today, someone pushed forward the baseline "to get access" to a version
of our qt port, instead of just updating the
"version>=". This happens All The Time.How to Work Inside the Current Baseline World
The way to make this work in the current world is very simple; you have a
baseline which is never updated past the first version of any library. This
results in the minimum version for everything, and only when one updates
the version>= will you get new versions.
This has two pain points, one minor and one major. The minor pain point is, of
course, when one adds a new port; that means one must also update all the
baselines to one which contain that port. This is frustrating, but can be worked
around.
The major pain point is significantly worse – this means we do not get any
warning or errors when adding a dependency without a version. It just gets to be
the old version without an error.
How I Want it to Work
I want to be able to depend on a registry, and require every single port to be
versioned, including internally to that registry. I want there to be an error if
the baseline doesn't exist, and the port is unversioned.
I do not mind if I have to explicitly ask for no baseline, for example, by doing
"baseline": null, but I want to be able to actually do this.